Introduction

Supervised Learning

In supervised learning, we provide the model with examples of features and labels. In a supervised setting, we have a perfect oracle that can tell the learner which labels are assigned to each input with the aim of generalizing the model to correctly operate on unseen data. The diagram below shows four examples of inputs mapped to outputs. Each of these examples will be provided to the model during training time, with a few examples set aside to test the models performance on new data.

For supervised learning to be effective, we need the training data to be properly formed, removing inconsistencies (such as one feature having multiple labels), and insuring that the training set is representative of the real-world problem we are trying to solve. Bad data is worse than no data, since we are can un-learn good weights with bad datum. When working with supervised learning, the data sets are often hand labeled by experts. In this scenario, human labelers can mistakenly introduce bias into their labels, or mis-label features in error. For this reason it is a good idea to test the quality of your training set.

You should be wary of four major issues when working with supervised learning:

Bias-variance tradeoff

Suppose we have multiple sets we are using for training, each equally representative of the real-world data we are modeling. An algorithm (recall that an algorithm is the device we use to train/fit a machine learning model) is biased toward a specific feature if the model is consistently wrong in the labeling of said feature. A learning algorithm has high variance if the model predicts different labels for the same feature. The prediction error of a model is the sum of the bias error and variance error. Unfortunately there is a tradeoff between bias and variance. If the model is too flexible (has too many trainable parameters), it will fit each piece of training data differently, increasing the variance. If the model is too rigid (not enough parameters), it will not fit the training data very well, increasing the bias. It is our job as Machine Learning engineers to find the model with just enough complexity to fit the data, but make it low enough in variance to predict unseen data.

Modeling Capacity

If the complexity of the prediction function is high, meaning we have numerous parameters that we are using to model the interaction between the features and labels, then we need large quantities of good data to optimize the weights of the "true" function (the predictor). On the other hand, when our prediction function is simple (say for instance a linear model), then we can often learn from a small amount of data.

Feature Dimensionality

Richard E. Bellman coined the phrase "curse of dimensionality" when referring to the ratio of data complexity to quantity. When out feature space has relatively few dimensions, then we do not need as much data. As the complexity of our input features grows, so too does the need for data. Take for example Natural Language Processing. Setting aside the problem of modeling capacity, it would be impossible to train a deep neural net to model human language off of only a few examples. Foundational models such as GPT-4 and Mixtral-8x7b require trillions of examples to properly model the complex patterns found in human language. When analyzing your training set, consider how much training data you have in comparison to the dimensionality of the features. As the dimensionality increases, the extra dimensions can throw off the algorithm, increasing the variance.

If you do not need as many dimensions to model your problem, then consider using a dimensionality reduction algorithm such as PCA or t-SNE.

Output Noise

Since humans are error-prone, hand labeled data should be checked for mislabeled features. Data that is mislabeled will introduce modeling errors, since the model will not know which label is correct. In this situation, it is better to remove the mislabeled points. No data is better than malformed data.

Overfitting

Trying to model the training data too closely can lead to overfitting. When you overfit a model it is not able to generalize to unseen data. A common example is when you use a higher-degree polynomial for your function. The higher degree polynomial will have more degrees of freedom to fit your training data, but will perform poorly on new data. Using a training set of examples that we feed to the model to learn from and a testing set to test the model at the end, we can test for overfitting. If we achieve a high accuracy on the training set (say 97%) and only a (34%) accuracy on the test set, then we have clearly tuned our model to only predict things it has seen before. Recall that a test set is not shown to the model until training has finished and it is ready to be evaluated. The test set contains data points that are representative of the problem we are trying to solve, but have been withheld from the model so we can test performance on new data.

Redundant Data

If the input features are highly correlated, then certain models perform worse. For instance Random Forest models are good at detecting interactions between features, but highly correlated data can mask these interactions¹.

Presence of Interactions and Non-linearities

Models that are linear perform well on data with linearities. If your data is complex, it likely is non-linear and will need some sort of mechanism to handle that interaction. Deep learning models with activation functions can handle non-linear data. We will cover that more when discussing activation functions.

Semi-supervised Learning

In semi-supervised learning you have a large number of un-labeled examples and a few labeled examples. This is a challenging paradigm, as it is far easier to work in the supervised learning domain where every example is labeled. Unfortunately, this is not often the case since labeling data is expensive and time consuming. Quickly hand labeling data leads to errors which is worse than having a set of high quality data. If data is abundant, yet labeling it all is too difficult then semi-supervised algorithms are ideal.

In the diagram above we can see a few points are labeled purple and others are pink. These points are the few examples of labeled data, where the white circles represent un-labeled examples that are more abundant. If we attempted to train a supervised model on the few examples that are labeled, we would likely produce a horizontal line that would cleanly separate the two sets. The problem is that line will mis-classify the points in the top set as being pink.

A better algorithm would be a clustering algorithm that (such as $k$-means) that would label nearby points. Keep in mind that the problem here is not the model complexity; a linear model would fail as would a non-linear model since the non-labeled data is not being used to train the model. The model separates the two sets in a way that is consistent with the train examples it is fed.

We will not cover semi-supervised learning in this series in any real detail. We plan to add more information on semi-supervised methods in another series in the future.

In Reinforcement Learning, an agent (a robot, an animal, or even you) interacts with an environment (a game, a system, or the world) through actions. The environment responds to these actions by providing the agent a reward signal and transitioning the state of the environment through the environment's mechanics.

An example of this is a self-driving car. In this scenario the agent is the car and the environment is the world. At every time interval the car makes a decision and takes an action. This action could be to continue driving straight, stop, make a left turn, etc. After the agent's action is taken, the environment provides some evolution of the world. For instance if the car drives straight, its location is now updated with a new location. The reward signal drives the agent toward a goal. In our scenario a reward signal can be the amount of gas expended at each time step (a reward can be negative, sometimes called a penalty), with a big payoff at the end when it reaches its destination. Deep Reinforcement Learning is out of the scope of this series and will be covered in a future series.